A Block Orthogonalization Procedure with Constant Synchronization Requirements

We propose an alternative orthonormalization method that computes the orthonor-mal basis from the right singular vectors of a matrix. Its advantage are: a) all operations are matrix-matrix multiplications and thus cache-eecient, b) only one synchronization point is required in parallel implementations, c) could be more stable than Gram-Schmidt. In addition, we consider the problem of incremental orthonormalization where a block of vectors is orthonormalized against a previously orthonormal set of vectors and among itself. We solve this problem by alternating iteratively between a phase of Gram-Schmidt and a phase of the new method. We provide error analysis and use it to derive bounds on how accurately the two successive orthonormalization phases should be performed to minimize total work performed. Our experiments connrm the favorable numerical behavior of the new method and its eeectiveness on modern parallel computers. AMS Subject Classiication. 65F15 1. Introduction. Computing an orthonormal basis from a given set of vectors is a basic computation, common in most scientiic applications. Often, it is also one of the most computationally demanding procedures because the vectors are of large dimension, and because the computation scales as the square of the number of vectors involved. Further, among several orthonormalization techniques the ones that ensure high accuracy are the more expensive ones. In many applications, orthonormalization occurs in an incremental fashion, where a new set of vectors (we call this internal set) is orthogonalized against a previously orthonormal set of vectors (we call this external), and then among themselves. This computation is typical in block Krylov methods, where the Krylov basis is expanded by a block of vectors 12, 11]. It is also typical when certain external orthogonalization constraints have to be applied to the vectors of an iterative method. Locking of converged eigenvectors in eigenvalue iterative methods is such an example 19, 22]. This problem diiers from the classical QR factorization in that the external set of vectors should not be modiied. Therefore, a two phase process is required; rst orthogonalizing the internal vectors against the external, and second the internal among themselves. Usually, the number of the internal vectors is much smaller than the external ones, and signiicantly smaller than their dimension. Another important diierence is that the accuracy of the R matrix of the QR factorization is not of primary interest, but rather the orthonormality of the produced vectors Q. A variety of orthogonalization techniques exist for both phases. …